DropSample: A New Training Method to Enhance Deep Convolutional Neural Networks for Large-Scale Unconstrained Handwritten Chinese Character Recognition

摘要

Inspired by the theory of Leitners learning box from the field of psychology,we propose DropSample, a new method for training deep convolutional neuralnetworks (DCNNs), and apply it to large-scale online handwritten Chinesecharacter recognition (HCCR). According to the principle of DropSample, eachtraining sample is associated with a quota function that is dynamicallyadjusted on the basis of the classification confidence given by the DCNNsoftmax output. After a learning iteration, samples with low confidence willhave a higher probability of being selected as training data in the nextiteration; in contrast, well-trained and well-recognized samples with very highconfidence will have a lower probability of being involved in the next trainingiteration and can be gradually eliminated. As a result, the learning processbecomes more efficient as it progresses. Furthermore, we investigate the use ofdomain-specific knowledge to enhance the performance of DCNN by adding a domainknowledge layer before the traditional CNN. By adopting DropSample togetherwith different types of domain-specific knowledge, the accuracy of HCCR can beimproved efficiently. Experiments on the CASIA-OLHDWB 1.0, CASIA-OLHWDB 1.1,and ICDAR 2013 online HCCR competition datasets yield outstanding recognitionrates of 97.33%, 97.06%, and 97.51% respectively, all of which aresignificantly better than the previous best results reported in the literature.